Inventorying items using image data

ABSTRACT

A method, system and computer-readable medium are provided for generating and maintaining a catalogue of belongings for a user. The method includes the steps of receiving image data from a user, the image data containing images of one or more items of belonging; identifying one or more items from the image data, determining information regarding the identified one or more items, and adding at least one of the identified one or more items and information regarding the at least one of the identified one or more items to a catalogue of belonging for the user.

CROSS-REFERENCE TO RELATED APPLICATION

The present application claims the benefit of U.S. Provisional PatentApplication Ser. No. 61/847,048, entitled “Inventorying Items UsingImage Data,” filed on Jul. 16, 2013, which is hereby incorporated byreference in its entirety for all purposes.

BACKGROUND Field

The present disclosure generally relates to insurance services, and moreparticularly to image processing for inventorying items.

Description of the Related Art

Personal property indexing services are available that assist a user ininventorying their belongings so that a record of these belongingsexists in the event of a loss pursued in an insurance claim.Unfortunately, very few people use the indexing services due to thetedium associated with individually cataloging each and every belongingowned, and thereafter maintaining that catalog.

SUMMARY

The disclosed system and method describe a method facilitating automaticgeneration and maintenance of a catalogue of user belongings. In certainembodiments, a method is provided for generating and maintaining acatalogue of belongings for a user. The method includes the steps ofreceiving image data from a user, the image data containing images ofone or more items of belonging. The method may further includeidentifying one or more items from the image data. The method mayfurther include determining information regarding the identified one ormore items. The method may further include adding at least one of theidentified one or more items and information regarding the at least oneof the identified one or more items to a catalogue of belonging for theuser, the catalogue of belongings.

In certain embodiments, a system for generating and maintaining acatalogue of belongings for a user is provided. The system may includeone or more processors and a machine-readable medium includinginstructions stored therein, which when executed by the processors,cause the processors to perform operations including receiving imagedata from a user, the image data containing images of one or more itemsof belonging located within a dwelling of the user. The operations mayfurther include identifying one or more items from the image data. Theoperations may further include determining if the user is associatedwith a catalogue of belongings. The operations may further includegenerating a catalogue of belongings for the user associated with thedwelling when the user is not associated with a catalogue of belongings.The operations may further include adding the one or more items to thegenerated catalogue.

In certain embodiments, a non-transitory computer-readable medium havingcomputer-executable instructions stored thereon for execution by aprocessor to perform a method generating and maintaining a catalogue ofbelongings for a user is provided. The method may include receivingimage data captured including items within a dwelling of a user. Themethod may further include identifying one or more items from the imagedata. The method may further include validating the one or more items bycomparing a location of the image data with a location of the dwelling.The method may further include identifying a catalogue of belongingsassociated with the user for cataloguing items within the dwelling ofthe user. The method may further include updating the catalogue ofbelongings according to the identified one or more items, such that thecatalogue of belongings stores information for each item captured withinthe received image data, wherein the updating includes one of adding,deleting or updating items stored within the catalogue of belongings.

It is understood that other configurations of the subject technologywill become readily apparent from the following detailed description,where various configurations of the subject technology are shown anddescribed by way of illustration. As will be realized, the subjecttechnology is capable of other and different configurations and itsseveral details are capable of modification in various other respects,all without departing from the scope of the subject technology.Accordingly, the drawings and detailed description are to be regarded asillustrative in nature and not as restrictive.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide furtherunderstanding and are incorporated in and constitute a part of thisspecification, illustrate disclosed embodiments and together with thedescription serve to explain the principles of the disclosedembodiments. In the drawings:

FIG. 1 illustrates an example client-server network environment, whichprovides for facilitating automatic generation and maintenance of acatalogue of user belongings.

FIG. 2 illustrates an example process for facilitating automaticgeneration and maintenance of a catalogue of user belongings.

FIG. 3 illustrates an example process for identifying one or more itemsbelonging to a user using images capture information.

FIG. 4 conceptually illustrates an electronic system with which someimplementations of the subject technology are implemented.

DETAILED DESCRIPTION

The detailed description set forth below is intended as a description ofvarious configurations of the subject technology and is not intended torepresent the only configurations in which the subject technology may bepracticed. The appended drawings are incorporated herein and constitutea part of the detailed description. The detailed description includesspecific details for the purpose of providing a thorough understandingof the subject technology. However, it will be clear and apparent thatthe subject technology is not limited to the specific details set forthherein and may be practiced without these specific details. In otherinstances, well-known structures and techniques have not been shown indetail so as not to obscure the disclosure.

Some examples provided herein describe identifying and/or storing userproperty information. In several examples, identification of suchinformation is performed after the user grants explicit permission forsuch user information to be identified and stored in memory. Each usermay be provided notice that such user information will be stored withsuch explicit consent from the user. The stored user information may beencrypted to protect user security.

The disclosed system and method facilitate generation and maintenance ofa catalogue of personal property or belongings (e.g. contents such asfurniture, electronics, home appliances, vehicles, bikes, bicycles,equipment, clothing, etc.) within a user dwelling (e.g., home or placeof business). In one example, the inventorying process for identifyingand cataloguing user personal belongings is automated. In some examples,one or more images (e.g., images or video) of user belongings arecaptured within the user dwelling and image recognition is used toprocess the captured images in order to index objects (or “items”) inthe images. As such, an automated indexing and generation of a catalogueof personal belongings of a user is provided.

FIG. 1 illustrates an example client-server network environment, whichprovides for facilitating automatic generation and maintenance of acatalogue of user belongings. A network environment 100 includes anumber of electronic devices 102, 104, 106 communicably connected to aserver 110 and one or more remote servers 120 by a network 108. Network108 can be a public communication network (e.g., the Internet, cellulardata network, dialup modems over a telephone network) or a privatecommunications network (e.g., private LAN, leased lines). Further,network 108 can include, but is not limited to, any one or more of thefollowing network topologies, including a bus network, a star network, aring network, a mesh network, a star-bus network, a tree or hierarchicalnetwork, and the like.

In some example implementations, electronic devices 102, 104, 106 can becomputing devices such as laptop or desktop computers, smartphones,PDAs, portable media players, tablet computers, image capture devices(e.g., camera or glasses), or other appropriate computing devices. Insome examples, electronic devices 102, 104, 106 have an image capturingcapability. In the example of FIG. 1, electronic device 102 is depictedas a smartphone, electronic device 104 is depicted as a desktop computerand electronic device 106 is depicted as a Personal Digital Assistant(PDA).

In some implementations, server 110 includes a processing device 112 anda data store 114. Processing device 112 executes computer instructionsstored in data store 114, for example, to facilitate automaticgeneration and maintenance of a catalogue of user belongings for usersinteracting with electronic devices 102, 104, 106. Server 110 mayfurther be in communication with remote servers 120 either through thenetwork 108 or through another network or communication means.

Remote servers 120 may be configured to perform various functionalitiesand/or storage capabilities described herein with regard to the server110 either alone or in combination with server 110. Server 110 maymaintain or be in communication with image recognition services,location services, and/or one or more financial or commerce applicationsor retail or financial establishments hosted on one or more remoteservers 120.

In some example aspects, server 110 and/or one or more remote servers120 can be a single computing device such as a computer server. In otherimplementations, server 110 and/or one or more remote servers 120 canrepresent more than one computing device working together to perform theactions of a server computer (e.g., cloud computing). Server 110 and/orone or more remote servers 120 may be coupled with various remotedatabases or storage services. While server 110 and the one or moreremote servers 120 are displayed as being remote from one another, itshould be understood that the functions performed by these servers maybe performed within a single server, or across multiple servers.

Communications between the client devices 102, 104, 106, server 110and/or one or more remote servers 120 may be facilitated through variouscommunication protocols. In some aspects, client devices 102, 104, 106,server 110 and/or one or more remote servers 120 may communicatewirelessly through a communication interface (not shown), which mayinclude digital signal processing circuitry where necessary. Thecommunication interface may provide for communications under variousmodes or protocols, including Global System for Mobile communication(GSM) voice calls, Short Message Service (SMS), Enhanced MessagingService (EMS), or Multimedia Messaging Service (MMS) messaging, CodeDivision Multiple Access (CDMA), Time Division Multiple Access (TDMA),Personal Digital Cellular (PDC), Wideband Code Division Multiple Access(WCDMA), CDMA2000, or General Packet Radio System (GPRS), among others.For example, the communication may occur through a radio-frequencytransceiver (not shown). In addition, short-range communication mayoccur, including using a Bluetooth, WiFi, or other such transceiver.

Users may interact with the system hosted by server 110, and/or one ormore social networking services hosted by remote servers 120, through aclient application installed at the electronic devices 102, 104, 106.Alternatively, the user may interact with the system and the one or moresocial networking services through a web based browser application atthe electronic devices 102, 104, 106. Communication between clientdevices 102, 104, 106 and the system, and/or one or more socialnetworking services, may be facilitated through a network (e.g., network108).

The network 108 can include, for example, any one or more of a personalarea network (PAN), a local area network (LAN), a campus area network(CAN), a metropolitan area network (MAN), a wide area network (WAN), abroadband network (BBN), the Internet, and the like. Further, thenetwork 108 can include, but is not limited to, any one or more of thefollowing network topologies, including a bus network, a star network, aring network, a mesh network, a star-bus network, tree or hierarchicalnetwork, and the like.

A user at a client device (e.g., electronic devices 102, 104, 106) mayaccess an application at the client device (e.g., hosted by server 110)and request to create or update a catalogue of belongings. In oneexample, the user is prompted to generate an account and/or log into anexisting account at the application. In one example, the user maycapture and input information (e.g., images or other information).

The entered information, in some examples, includes images captured, forexample, using the client device or other device and uploaded to theapplication. In one example, the images may be captured by the useroperating the client device. In another example, the user may beprompted to grant access to a live agent to remotely control theapplication and/or client device in order to capture the images.

In one example, the client device may be configured to enable remotecontrol of certain functions of the client device by a remote agent(e.g., a live agent through audio or video conference). For example, theclient application running on the client device may be configured toaccess various capabilities of the client device and/or capabilities ofdevices in communication with the client device. For example, theapplication may have access to one or more cameras on the client device(e.g., a front-facing camera and/or a rear-facing camera), microphoneson the client device, or other sensors or components on the device(e.g., an accelerometer, infrared sensors, a global positioning system,a light, etc.). The client application may also provide a way for theagent to access and control the various functions or capabilitiesthrough the client device and/or a system (e.g., operating system, orother application system) or processor at the client device.

In some examples, the client device may provide an interface and/orcapability for the agent to control certain capabilities of the clientdevice (e.g., remote control). The agent, for example, may request thatthe client device capture an image, activate a flash when capturing animage, turn on a light on the client device, zoom in the camera, focusthe camera, adjust an exposure setting on the camera, or switch a viewon a display on the client device.

The switching of the view on the display of the client device mayinclude switching the view to at least one of a view of what afront-facing camera is detecting, a view of what a rear-facing camera isdetecting, or a video conferencing view of a remote camera (e.g., a viewof a camera at and/or in communication with the client device or a viewof the live agent). In additional aspects, switching a view may includechanging the display on the client device to a portrait view, alandscape view, or a picture-in-picture view of multiple views.

In response to receiving the captured images, the server 110 mayidentify one or more items within the captured image using imagerecognition. In one example, the server 110 may have image recognitioncapability and/or may be in communication with an image recognitionservice (e.g., hosted at one or more remote servers 120).

In one example, geographic information embedded in the captured image orotherwise accessible through the image capturing device (e.g., byquerying a location service) is retrieved to verify the location of theimages being captured. In one example, the user belongings areassociated with the user dwelling and the catalogue is generated forinsurance purposes. In such examples, the verification of the geographiclocation of the images as being within the user dwelling allows for acatalogue of users belongings within a dwelling, for example, for futureuse in case of damage and/or loss. In one example, a list of recognizeditems from the captured image(s) is provided to the user at the clientdevice. The user may further enter additional information for each item,including for example, date of purchase, any special characteristicssuch as upgrades or damages to the item, item price, etc. Theinformation is associated with item. In another example, one or more ofsuch information may be automatically retrieved and associated with theitem. The one or more items are then added to a catalogue of userbelongings.

In some examples, the login information for the user is used at server110 to determine if the user is associated with an existing list ofbelongings (“a catalogue”). If so, the one or more items may be comparedwith the existing items with the catalogue and items may be added to thecatalogue and/or existing information regarding items may be updatedwithin the existing catalogue. Otherwise, if the user is not associatedwith an existing catalogue, the server 110 may generate a catalogue ofthe items recognized from the user information entered during thesession (e.g., from images and/or additional information) and may storethe items in association with the user.

FIG. 2 illustrates an example process 200 for facilitating automaticgeneration and maintenance of a catalogue of user belongings. In step201, an indication of a request to add one or more items to a catalogueof belongings associated with a user is received. In one example, theuser may capture an image (e.g., still images or video) containing oneor more items. In one example, when the user logs into the system,prompts may be provided to the user, to allow the user to capture imagesof items within different areas of the dwelling to generate a fullcatalogue of the user belongings. In addition, items may be entered bydetecting item purchases by the user (e.g., through detecting purchasesat one or more accounts associated with the user account within thesystem). In yet another example, information regarding items may beentered by scanning or capturing an image of a purchase receipt by theuser.

Upon receiving the request, in step 202, the one or more items areidentified. In one example, where the request include one or more imagesuploaded to the system, the system may provide the images to an imagerecognition engine to identify the items within each image. FIG. 3,described in more details below, illustrates an example process forgenerating a listing of one or more items from one or more image datacaptured by a user. In one example, the items may also include itemsreceived from one or more other services (e.g., purchases detectedthrough accounts linked with the user profile), and/or entered throughother means such as scanning in item barcodes or receipts and/or bymanually entering the item(s).

In one example, upon identifying one or more items, the user may beprompted to enter additional information regarding the identified itemsand/or to verify that the identified items are correct. In one example,the items are associated with the corresponding additional informationentered by the user.

As described above, once the list of one or more items is identified,the one or more items within the list may be validated, for example,using geographic location information. In one example, images mayinclude geo tags that indicate the location of the images. In oneexample, the location may be determined using location indicators of thedevice used to capture the image and/or upload the image to the system.In one example, the location of the images and/or device is comparedagainst the location of the user dwelling to confirm that the itemsbeing captured are in fact items within the user dwelling.

In some examples, in addition to object recognition, the system (e.g.,and/or one or more applications and/or systems in communication with thesystem) may further provide a value for each identified item. In someexamples, the value of the item may be determined using information fromone or more sources. For example, the value may be received as anadditional input from the user. Item information may include identifierinformation such as brand, ID, model, characteristics (e.g., size, type,etc.), purchase date, etc. The item information may be determinedthrough object recognition, a search engine and/or user input. In oneexample, the value of an item is determined using item information. Forexample, a search of the item, including the image and/or informationregarding the item, may be performed to find same or similar itemshaving a known value. The value of the same or similar item may then beused to associate a value with the item of the one or more items. Thevalue may be adjusted according to information such as date of purchase(e.g., depreciation) and/or damages or improvements of the item. Inanother example, a time stamp associated with the capture and/or uploadof the image may also be used for determining information regarding theitem including for example, date of purchase, model, or value of theitem.

In step 203, it is determined if the user is associated with an existingcatalogue of belongings. If not, in step 204, the one or more itemsidentified in step 202 including information regarding each item arestored as a catalogue in association with the user (e.g., in associationwith the user profile). Otherwise, the process continues to step 205,and the existing catalogue associated with the user is retrieved. Instep 206, the one or more items identified in step 202 are compared tothe one or more items of the existing catalogue retrieved in step 205.

In one example, each item of the identified one or more items iscompared to the items within the existing catalogue to determine if eachof the one or more items is present within the existing catalogue and/orthe item and/or information has been modified. In one example, themodification may include additional or modified information beingprovided regarding the item. In one example, the modification mayinclude a new image of the item being captured and/or informationregarding the item being updated according to the captured image (e.g.,improvements, damage, etc.).

In step 207, one or more new or modified items of the one or more itemsidentified in step 202 are determined according to the comparison instep 206. In step 208, the one or more new or modified items are addedto the existing catalogue associated with the user. In one example, inaddition to adding or modifying items within the list, the system mayidentify one or more items previously included in the existing catalogueof the user which were not identified in step 202 and may remove theitems from the existing catalogue of the user.

Generating and maintaining a catalogue in this manner may be beneficialin facilitating a catalogue for each user which may be used fordetermining an amount of insurance coverage (e.g., contents insurancecoverage) and/or for claim purposes in case of loss or damage ofproperty. In one example, since the catalogue is generated andmaintained prior to the loss or damage, the time stamp associated withthe items being added to a catalogued may be used as evidence that theitem was owned by the user. In one example, the catalogue allows forquick claim times as the user may satisfy their duty of providing a fulllist of belongings by providing the catalogue along with their claim.

FIG. 3 illustrates an example process 300 for identifying one or moreitems belonging to a user using images capture information. In step 301,one or more image data captured by a user is received. The image data,may, for example include images and/or video captured using an imagecapture device (e.g., a smart phone, camera, etc.). In one example, asdescribed above, the user may log into the catalogue system and may beprompted to capture images and/or video of the user belongings forcataloguing purposes, one example, the images may be captured forgenerating and/or updating a catalogue associated with the user.

In step 302, the captured image data is compared to a reference list ofitems to recognize one or more items within the received image data. Inone example, various object recognition techniques may be used torecognize items within each image or video. In one example, step 302 maybe performed by sending a request to a remote object recognition engine.In another example, the system may store a reference list of itemslocally, and object recognition may be performed locally.

Based on the comparison, in step 303, one or more items are identified.In one example, each identified item corresponds to a belonging of theuser within the user dwelling (e.g., residence, business or otherlocation where the images have been captured). As described above,location information regarding the image capture data may be determinedto validate that each item is present at the dwelling. In one example,for each identified item, the comparison may return various informationincluding brand, ID, model, characteristics (e.g., size, type, etc.), orother identifiers or characteristics of the item. In another example,additional information regarding an item may be retrieved using theimage and/or information retrieved according to the object recognitiontechniques. For example, a value associated with the item may beretrieved. In one example, the object recognition and/or informationdetermination for each item may be performed in real time whereinformation regarding the items is provided to the user after the imagedata is captured and/or uploaded into the system. In another example,one or more of the identification and/or information retrieval may beperformed at a later time and information may be presented to the useras it becomes available.

In one example, the reference list of items may be updated with therecognized image data and item information. In this manner, informationregarding captured image data, and corresponding item information may bestored for later use when generating and maintaining catalogues for theuser or one or more other users.

In step 304, a listing of one or items identified according to thecomparison is provided for display to the user. As described above, thelist may include the items recognized from the image data and/orinformation regarding the items including identifier information,characteristics information and/or value information of the item.

In step 305, the user may enter additional information regarding theitem (e.g., notations, descriptions, etc.), including for example,characteristics information, purchase information and/or valueinformation. In one example, the information entered by the user isprovided to complete the information fields for each item that were notfilled in automatically in response to the image recognition and otherinformation available and/or to supplement such information.

In step 306, the listing of one or more items is updated using theinformation provided by the user. For example, the user may add one ormore items, may modify or add additional information regarding one ormore items, or may provide additional information used to modifyinformation regarding the item (e.g., value indications, date ofpurchase indications, etc.).

In one example, where information regarding the recognized items isavailable, the system may facilitate providing the information fordisplay to the user in a live manner. For example, in one instance, anaugmented reality may be generated where information regarding each itemis provided to the user as the user captures images of the item using adevice.

Many of the above-described features and applications are implemented assoftware processes that are specified as a set of instructions recordedon a computer readable storage medium (also referred to as computerreadable medium). When these instructions are executed by one or moreprocessing unit(s) (e.g., one or more processors, cores of processors,or other processing units), they cause the processing unit(s) to performthe actions indicated in the instructions. Examples of computer readablemedia include, but are not limited to, CD-ROMs, flash drives, RAM chips,hard drives, EPROMs, etc. The computer readable media does not includecarrier waves and electronic signals passing wirelessly or over wiredconnections.

In this specification, the term “software” is meant to include firmwareresiding in read-only memory or applications stored in magnetic storage,which can be read into memory for processing by a processor. Also, insome implementations, multiple software aspects of the subjectdisclosure can be implemented as sub-parts of a larger program whileremaining distinct software aspects of the subject disclosure. In someimplementations, multiple software aspects can also be implemented asseparate programs. Finally, any combination of separate programs thattogether implement a software aspect described here is within the scopeof the subject disclosure. In some implementations, the softwareprograms, when installed to operate on one or more electronic systems,define one or more specific machine implementations that execute andperform the operations of the software programs.

A computer program (also known as a program, software, softwareapplication, script, or code) can be written in any form of programminglanguage, including compiled or interpreted languages, declarative orprocedural languages, and it can be deployed in any form, including as astandalone program or as a module, component, subroutine, object, orother unit suitable for use in a computing environment. A computerprogram may, but need not, correspond to a file in a file system. Aprogram can be stored in a portion of a file that holds other programsor data (e.g., one or more scripts stored in a markup languagedocument), in a single file dedicated to the program in question, or inmultiple coordinated files (e.g., files that store one or more modules,sub programs, or portions of code). A computer program can be deployedto be executed on one computer or on multiple computers that are locatedat one site or distributed across multiple sites and interconnected by acommunication network.

FIG. 4 conceptually illustrates an electronic system with which someimplementations of the subject technology are implemented. Electronicsystem 400 can be a server, computer, phone, PDA, laptop, tabletcomputer, television, image capture devices (e.g., cameras, glasses,etc.), with one or more processors embedded therein or coupled thereto,or any other sort of electronic device. Such an electronic systemincludes various types of computer readable media and interfaces forvarious other types of computer readable media. Electronic system 400includes a bus 408, processing unit(s) 412, a system memory 404, aread-only memory (ROM) 410, a permanent storage device 402, an inputdevice interface 414, an output device interface 406, and a networkinterface 416.

Bus 408 collectively represents all system, peripheral, and chipsetbuses that communicatively connect the numerous internal devices ofelectronic system 400. For instance, bus 408 communicatively connectsprocessing unit(s) 412 with ROM 410, system memory 404, and permanentstorage device 402.

From these various memory units, processing unit(s) 412 retrievesinstructions to execute and data to process in order to execute theprocesses of the subject disclosure. The processing unit(s) can be asingle processor or a multi-core processor in different implementations.

ROM 410 stores static data and instructions that are needed byprocessing unit(s) 412 and other modules of the electronic system.Permanent storage device 402, on the other hand, is a read-and-writememory device. This device is a non-volatile memory unit that storesinstructions and data even when electronic system 400 is off. Someimplementations of the subject disclosure use a mass-storage device(such as a magnetic or optical disk and its corresponding disk drive) aspermanent storage device 402.

Other implementations use a removable storage device (such as a floppydisk, flash drive, and its corresponding disk drive) as permanentstorage device 402. Like permanent storage device 402, system memory 404is a read-and-write memory device. However, unlike storage device 402,system memory 404 is a volatile read-and-write memory, such a randomaccess memory. System memory 404 stores some of the instructions anddata that the processor needs at runtime. In some implementations, theprocesses of the subject disclosure are stored in system memory 404,permanent storage device 402, and/or ROM 410. For example, the variousmemory units include instructions for generating and maintaining acatalogue of belongings for a user according to various implementations.From these various memory units, processing unit(s) 412 retrievesinstructions to execute and data to process in order to execute theprocesses of some implementations.

Bus 408 also connects to input and output device interfaces 414 and 406.Input device interface 414 enables the user to communicate informationand select commands to the electronic system. Input devices used withinput device interface 414 include, for example, alphanumeric keyboardsand pointing devices (also called “cursor control devices”). Outputdevice interfaces 406 enables, for example, the display of imagesgenerated by the electronic system 400. Output devices used with outputdevice interface 406 include, for example, printers and display devices,such as cathode ray tubes (CRT) or liquid crystal displays (LCD). Someimplementations include devices such as a touchscreen that functions asboth input and output devices.

Finally, as shown in FIG. 4, bus 408 also couples electronic system 400to a network (not shown) through a network interface 416. In thismanner, the computer can be a part of a network of computers (such as alocal area network (“LAN”), a wide area network (“WAN”), or an Intranet,or a network of networks, such as the Internet. Any or all components ofelectronic system 400 can be used in conjunction with the subjectdisclosure.

These functions described above can be implemented in digital electroniccircuitry, in computer software, firmware or hardware. The techniquescan be implemented using one or more computer program products.Programmable processors and computers can be included in or packaged asmobile devices. The processes and logic flows can be performed by one ormore programmable processors and by one or more programmable logiccircuitry. General and special purpose computing devices and storagedevices can be interconnected through communication networks.

Some implementations include electronic components, such asmicroprocessors, storage and memory that store computer programinstructions in a machine-readable or computer-readable medium(alternatively referred to as computer-readable storage media,machine-readable media, or machine-readable storage media). Someexamples of such computer-readable media include RAM, ROM, read-onlycompact discs (CD-ROM), recordable compact discs (CD-R), rewritablecompact discs (CD-RW), read-only digital versatile discs (e.g., DVD-ROM,dual-layer DVD-ROM), a variety of recordable/rewritable DVDs (e.g.,DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD cards, mini-SDcards, micro-SD cards, etc.), magnetic and/or solid state hard drives,read-only and recordable Blu-Ray® discs, ultra density optical discs,any other optical or magnetic media, and floppy disks. Thecomputer-readable media can store a computer program that is executableby at least one processing unit and includes sets of instructions forperforming various operations. Examples of computer programs or computercode include machine code, such as is produced by a compiler, and filesincluding higher-level code that are executed by a computer, anelectronic component, or a microprocessor using an interpreter.

While the above discussion primarily refers to microprocessor ormulti-core processors that execute software, some implementations areperformed by one or more integrated circuits, such as applicationspecific integrated circuits (ASICs) or field programmable gate arrays(FPGAs). In some implementations, such integrated circuits executeinstructions that are stored on the circuit itself.

As used in this specification and any claims of this application, theterms “computer”, “server”, “processor”, and “memory” all refer toelectronic or other technological devices. These terms exclude people orgroups of people. For the purposes of the specification, the termsdisplay or displaying means displaying on an electronic device. As usedin this specification and any claims of this application, the terms“computer readable medium” and “computer readable media” are entirelyrestricted to tangible, physical objects that store information in aform that is readable by a computer. These terms exclude any wirelesssignals, wired download signals, and any other ephemeral signals.

To provide for interaction with a user, implementations of the subjectmatter described in this specification can be implemented on a computerhaving a display device, e.g., a CRT (cathode ray tube) or LCD (liquidcrystal display) monitor, for displaying information to the user and akeyboard and a pointing device, e.g., a mouse or a trackball, by whichthe user can provide input to the computer. Other kinds of devices canbe used to provide for interaction with a user as well; for example,feedback provided to the user can be any form of sensory feedback, e.g.,visual feedback, auditory feedback, or tactile feedback; and input fromthe user can be received in any form, including acoustic, speech, ortactile input. In addition, a computer can interact with a user bysending documents to and receiving documents from a device that is usedby the user; for example, by sending web pages to a web browser on auser's client device in response to requests received from the webbrowser.

Implementations of the subject matter described in this specificationcan be implemented in a computing system that includes a back endcomponent, e.g., as a data server, or that includes a middlewarecomponent, e.g., an application server, or that includes a front endcomponent, e.g., a client computer having a graphical user interface ora Web browser through which a user can interact with an implementationof the subject matter described in this specification, or anycombination of one or more such back end, middleware, or front endcomponents. The components of the system can be interconnected by anyform or medium of digital data communication, e.g., a communicationnetwork. Examples of communication networks include a local area network(“LAN”) and a wide area network (“WAN”), an inter-network (e.g., theInternet), and peer-to-peer networks (e.g., ad hoc peer-to-peernetworks).

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other. In someimplementations, a server transmits data (e.g., an HTML page) to aclient device (e.g., for purposes of displaying data to and receivinguser input from a user interacting with the client device). Datagenerated at the client device (e.g., a result of the user interaction)can be received from the client device at the server.

In the previous detailed description, numerous specific details havebeen set forth to provide a full understanding of the presentdisclosure. It will be apparent, however, to one ordinarily skilled inthe art that embodiments of the present disclosure may be practicedwithout some of the specific details. In other instances, well-knownstructures and techniques have not been shown in detail so as not toobscure the disclosure.

The previous description is provided to enable any person skilled in theart to practice the various aspects described herein. While theforegoing has described what are considered to be the best mode and/orother examples, it is understood that various modifications to theseaspects will be readily apparent to those skilled in the art, and thegeneric principles defined herein may be applied to other aspects. Thus,the claims are not intended to be limited to the aspects shown herein,but is to be accorded the full scope consistent with the languageclaims, wherein reference to an element in the singular is not intendedto mean “one and only one” unless specifically so stated, but rather“one or more.” Unless specifically stated otherwise, the terms “a set”and “some” refer to one or more. Pronouns in the masculine (e.g., his)include the feminine and neuter gender (e.g., her and its) and viceversa. Headings and subheadings, if any, are used for convenience onlyand do not limit the invention.

It is understood that the specific order or hierarchy of steps in theprocesses disclosed is an illustration of exemplary approaches. Basedupon design preferences, it is understood that the specific order orhierarchy of steps in the processes may be rearranged. Some of the stepsmay be performed simultaneously. The accompanying method claims presentelements of the various steps in a sample order, and are not meant to belimited to the specific order or hierarchy presented.

Terms such as “top,” “bottom,” “front,” “rear” and the like as used inthis disclosure should be understood as referring to an arbitrary frameof reference, rather than to the ordinary gravitational frame ofreference. Thus, a top surface, a bottom surface, a front surface, and arear surface may extend upwardly, downwardly, diagonally, orhorizontally in a gravitational frame of reference.

A phrase such as an “aspect” does not imply that such aspect isessential to the subject technology or that such aspect applies to allconfigurations of the subject technology. A disclosure relating to anaspect may apply to all configurations, or one or more configurations. Aphrase such as an aspect may refer to one or more aspects and viceversa. A phrase such as an “embodiment” does not imply that suchembodiment is essential to the subject technology or that suchembodiment applies to all configurations of the subject technology. Adisclosure relating to an embodiment may apply to all embodiments, orone or more embodiments. A phrase such an embodiment may refer to one ormore embodiments and vice versa.

The word “exemplary” is used herein to mean “serving as an example orillustration.” Any aspect or design described herein as “exemplary” isnot necessarily to be construed as preferred or advantageous over otheraspects or designs.

All structural and functional equivalents to the elements of the variousaspects described throughout this disclosure that are known or latercome to be known to those of ordinary skill in the art are expresslyincorporated herein by reference and are intended to be encompassed bythe claims. Moreover, nothing disclosed herein is intended to bededicated to the public regardless of whether such disclosure isexplicitly recited in the claims. No claim element is to be construedunder the provisions of 34 U.S.C. § 112, sixth paragraph, unless theelement is expressly recited using the phrase “means for” or, in thecase of a method claim, the element is recited using the phrase “stepfor.” Furthermore, to the extent that the term “include,” “have,” or thelike is used in the description or the claims, such term is intended tobe inclusive in a manner similar to the term “comprise” as “comprise” isinterpreted when employed as a transitional word in a claim.

1-20. (canceled)
 21. A method comprising: receiving, by a firstcomputing device and from a second computing device associated with auser, one or more images; performing, by the first computing device,object recognition on the one or more images to identify one or moreitems within the one or more images; validating, by the first computingdevice and based on first geographical data associated with the one ormore images and second geographical data associated with the user, theone or more items; determining, by the first computing device,additional data associated with the one or more items; and storing, bythe first computing device, data identifying the one or more items andthe additional data in a catalogue associated with the user.
 22. Themethod of claim 21, wherein the first geographical data comprises one ormore geo tags within the one or more images.
 23. The method of claim 21,wherein the second geographical data comprises locational dataassociated with a dwelling of the user.
 24. The method of claim 21,wherein the second geographical data comprises locational dataassociated with the second computing device.
 25. The method of claim 21,wherein the additional data is received from the second computing deviceand comprises one or more characteristics of each item of the one ormore items.
 26. The method of claim 21, wherein the object recognitionis performed in real-time as the one or more images are received by thefirst computing device.
 27. The method of claim 21, further comprising:providing, by the first computing device and to the second computingdevice, a list of the one or more items for display on the secondcomputing device.
 28. An apparatus comprising: one or more processors;and memory storing instructions which, when executed by the one or moreprocessors, cause the apparatus to: receive, from a first computingdevice associated with a user, one or more images; perform objectrecognition on the one or more images to identify one or more itemswithin the one or more images; validate, based on first geographicaldata associated with the one or more images and second geographical dataassociated with the user, the one or more items; receive additional dataassociated with the one or more items; and store data identifying theone or more items and the additional data in a catalogue associated withthe user.
 29. The apparatus of claim 28, wherein the first geographicaldata comprises one or more geo tags within the one or more images. 30.The apparatus of claim 28, wherein the second geographical datacomprises locational data associated with the first computing device.31. The apparatus of claim 28, wherein the additional data is receivedfrom the first computing device and comprises one or morecharacteristics of each item of the one or more items.
 32. The apparatusof claim 28, wherein the object recognition is performed in real-time asthe one or more images are received.
 33. The apparatus of claim 28,memory storing instructions which, when executed by the one or moreprocessors, cause the apparatus to: provide, to the first computingdevice, a list of the one or more items for display on the firstcomputing device.
 34. The apparatus of claim 28, memory storinginstructions which, when executed by the one or more processors, causethe apparatus to: generate the catalogue based on information receivedfrom the first computing device.
 35. One or more non-transitory computerreadable media storing instructions which, when executed by one or moreprocessors, cause the one or more processors to: receive, from a firstcomputing device associated with a user, one or more images; performobject recognition on the one or more images to identify one or moreitems within the one or more images; validate, based on firstgeographical data associated with the one or more images and secondgeographical data associated with the user, the one or more items;receive additional data associated with the one or more items; and storedata identifying the one or more items and the additional data in acatalogue associated with the user.
 36. One or more non-transitorycomputer readable media of claim 35, wherein the first geographical datacomprises one or more geo tags within the one or more images.
 37. One ormore non-transitory computer readable media of claim 35, wherein thesecond geographical data comprises locational data associated with thefirst computing device.
 38. One or more non-transitory computer readablemedia of claim 35, wherein the additional data is received from thefirst computing device and comprises one or more characteristics of eachitem of the one or more items.
 39. One or more non-transitory computerreadable media of claim 35, wherein the object recognition is performedin real-time as the one or more images are received.
 40. One or morenon-transitory computer readable media of claim 35, storing instructionswhich, when executed by the one or more processors, cause the one ormore processors to: provide, to the first computing device, a list ofthe one or more items for display on the first computing device.